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SUMMARY 

Knowledge  acquisition  is  commonly  regarded  as  the  major  bottle-neck  in  Expert  System 
(ES)  development.  Numerous  techniques  and  commercial  packages  have  been  built  in 
order  to  widen  this  bottle-neck.  Unfortunately,  these  have  had  little  success,  except  in 
simple,  highly  confined  domains,  therefore  knowledge  acquisition  is  still  regarded  as  a 
"black  art".  This  paper  critically  analyses  current  acquisition  techniques  and  suggests  future 
research  directions. 
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1  Introduction 

An  ES  embodies  a  human  expert’s  domain-specific  knowledge,  judgment  and  experience,  the 
acquisition  of  which  is  refined  over  a  number  of  years  and  forms  an  integral  part  of  ES 
development.  Knowledge  acquisition  techniques  have  evolved  into  three  major  categories, 
the  most  commonly  used  being  the  highly  iterative  articulation  approach.  This  involves 
interaction  between  the  knowledge  engineer  and  an  expert  (or  group  of  experts),  in  which  a 
two  tier  hierarchy  (the  level  of  reasoning,  and  the  level  of  underlying  structure  and  function) 
is  iteratively  acquired.  The  result  is  a  model  or  schema  encompassing  what  is  thought  of  as 
relevant  knowledge. 

In  contrast  to  the  articulation  approach,  automated  rule  induction  algorithms  [Quinlan83] 
have  been  devised  whereby  the  expert  supplies  a  number  of  case  studies  by  way  of  domain 
specific  attributes,  values  and  concluding  hypotheses.  These  case  studies  are  then  analysed 
and  a  set  of  rules  are  induced. 

The  third  approach  is  shell  automated  knowledge  acquisition.  This  is  basically  a  systematic 
top-down  question  and  answer  process  in  which  a  hierarchical  rule-based  structure  is  created. 

In  section  2,  ES  knowledge-base  formalisms  are  defined.  Section  3  then  describes  knowledge 
acquisition  by  articulation.  Rule  induction  theory  is  discussed  in  section  4,  after  which 
section  5  outlines  automated  acquisition  techniques.  Finally,  section  6  indicates  future 
directions  based  upon  the  limitations  of  the  discussed  approaches. 


2  Expert  system  Knowledge  -  base  formalisms 

A  knowledge-base  is  a  model  of  a  human's  understanding  of  a  highly  confined  domain  of 
expertise.  Knowledge  acquisition  is  a  methodology  for  extracting  this  expertise,  from  which 
a  software  model  can  be  constructed.  The  developed  model  will  generally  consist  of  numerous 
types  of  knowledge  whose  function,  for  example,  may  be  that  of;  disregarding  or  supporting 
hypotheses,  emulating  the  structure  of  class  hierarchies  or  defining  the  search  strategies 
applicable  to  the  overall  domain  of  expertise. 

Before  attempting  to  acquire  domain  specific  knowledge,  it  is  essential  that  the  knowledge 
engineer  fully  understands  the  types  of  knowledge  that  can  be  uncovered.  In  addition,  he  must 
also  be  able  to  identify  and  construct  relational  models  between  the  various  types  of 
knowledge  formalisms  that  have  been  discovered.  Knowledge-based  formalisms  generally 
comprise  a  four  tier  hierarchy,  the  components  of  which  are  commonly  known  as:  Structural 
Knowledge,  Empirical  Associations,  Causal  Models  and  Strategic  Knowledge. 

2.1  Structural  Knowledge 

Structural  Knowledge  is  the  lowest  level  of  the  knowledge  classification  hierarchy 
and  consists  of  concepts  easily  obtainable  from  manuals  and  books.  These  concepts  may 
include  classification  information,  basic  interconnections  and  interrelations  between  a 
domain's  subclasses  [Johnson83]. 

2.2  Empirical  Associations 

Empirical  Associations  are  heuristic  rules.  They  may  determine  what  actions  should  be 
carried  out  in  certain  situations,  what  lines  of  reasoning  can  be  disregarded  and  what 
conclusions  may  be  drawn  from  the  evidence  supplied  [Rouse80].  This  knowledge  level 
is  based  upon  an  expert's  opinions  and  experience.  For  example,  other  experts  may  have 
conflicting  views  about  this  empirical  knowledge  (not  usually  written  in  text  books)  and 
it  may  be  probabilistic  rather  than  deterministic. 
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2.3  Causal  Models 

Causal  Models  are  collections  of  interrelated  rules  usually  associated  with  a  particular 
problem,  hypothesis  or  overall  diagnostic  conclusion.  These  rules  interact  in  such  a  way 
as  to  solve  problems  by  searching  Causal  Chains,  therefore  solutions  are  determined  by 
inferring  cause-effect  relationships  [Winston84]. 

2.4  Strategic  Knowledge 

Strategic  knowledge  uses  strategies  to  choose  optimum  search  spaces  by  the  use  of  meta¬ 
rules  [SterlingSO],  that  guide  a  search  down  a  particular  path  (when  several  paths 
exist)  before  considering  any  other  path.  This  form  of  knowledge  is  basically  the 
inference  and  search  guidance  mechanism  and  is  sometimes  referred  to  as  "knowledge 
about  knowledge". 

3  Knowledge  Acquisition  by  Articulation 

There  are  many  reported  knowledge  acquisition  articulation  approaches  and  case  studies. 
These  ad-hoc  approaches  have  been  developed  to  elicit  rules,  heuristics  and  general  domain 
knowledge.  Furthermore,  they  cannot  readily  identify  the  expert’s  detailed  reasoning  or 
behaviour. 

Knowledge  acquisition  by  articulation  relies  upon  the  interviewing  skills  and  thought 
processes  of  the  knowledge  engineer.  The  willingness  and  communication  skills  of  the  expert 
also  play  a  major  role  in  the  success  of  this  approach.  These  requirements  are  often  far  from 
satisfied  during  a  typical  knowledge  engineer/expert  consultation,  consequently,  the 
effectiveness  of  articulation  is  limited  but  it  can  be  a  useful  technique  for  rapid  prototyping. 
As  a  result,  many  iterations  of  knowledge-base  construction /analysis  may  be  required  before 
the  expert's  domain  of  expertise  is  captured  to  an  acceptable  level  of  competence. 

3.1  Concept  Sorting 

Concept  Sorting  [ChiSl]  involves  extracting  a  group  of  concepts  from  journals,  books  or 
glossaries.  Each  concept  is  written  on  an  individual  card  which  is  then  categorised  by 
an  expert  who  describes  the  relationships  between  them.  Concept  sorting  simplifies 
the  task  of  knowledge  organisation  and  forms  the  basis  for  producing  a  hierarchical 
conceptualization  of  the  domain  structure.  The  success  of  this  technique  relies  upon  the 
completeness  of  the  concepts  originally  discovered  and  written  on  the  cards.  There  is  a 
high  probability  that  some  concepts  will  be  missing  or  irrelevant  initially,  since  this  is 
done  by  the  knowledge  engineer. 

3.2.  Critical  Incident 

The  Critical  Incident  memory  probing  technique  requires  the  expert  to  focus  upon 
particularly  memorable  events  that  require  a  great  deal  of  expertise  [Wellbank83]. 
Because  the  selected  cases  are  unusual,  it  is  likely  that  they  will  be  easy  to  recall  and 
they  have  the  added  benefit  of  stimulating  the  interest  of  the  expert.  The  selected 
cases  are  usually  extremely  difficult  or  remarkable,  therefore  it  is  possible  to  elicit 
hidden  heuristics  and  complex  rule  interrelationships. 

This  approach  relies  upon  the  accurate  recollection  of  the  expert's  thoughts  and 
behaviour  at  the  time  of  the  event.  In  practice,  it  has  been  found  that  most  experts 
digress,  over-generalise  and  confuse  issues  with  irrelevant  statements  and  facts.  Hence, 
care  must  be  taken  in  the  analysis  of  the  acquired  knowledge,  also  other  elicitation 
techniques  must  be  used  to  clarify  poorly  explained  and  ambiguous  areas. 
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3.3.  Conversation  Theory  and  Focussed  Interviewing 

Conversation  Theory  [JohnsonSS]  and  Focussed  Interviewing  [Breuker84]  are  two 
elicitation  techniques  with  many  similarities.  They  both  resemble  normal 
conversation  in  which  the  knowledge  engineer  probes  the  domain  expert  with  pre¬ 
determined  topics  and  concepts.  The  effectiveness  of  this  approach  depends  upon  the 
personality  and  probing  techniques  employed  by  the  knowledge  engineer  who  should 
ideally  be  proficient  in  interview  protocols  so  that  the  expert  feels  relaxed  and 
un  threatened. 

Verbal  probes  and  body  language  are  an  integral  part  of  interviewing,  for  instance  a 
simple  "yes"  or  nod  of  the  head  will  encourage  the  expert  to  continue  along  the  current 
train  of  thought.  Conversely,  if  the  interviewer  shakes  his  head  or  says  "sorry  I  don't 
understand",  then  this  will  invoke  another  explanation  or  it  will  direct  the  expert 
along  another  route. 

Conversation  Theory  explores  the  expert's  domain  at  two  levels:  level  0  and  level  1. 
Level  0  analysis  is  aimed  at  eliciting  general  knowledge  such  as  descriptions  of 
procedures  and  algorithms,  whereas  level  1  analysis  is  at  a  deeper  level  (mainly 
causal  relationships)  with  emphasis  upon  explanations  of  why  level  0  procedures 
work.  This  approach  requires  that  the  knowledge  engineer  employs  "teachback",  by 
the  use  of  verbal  probes  and  body-language,  to  check  his  understanding  of  what  has 
been  said.  As  a  result,  interviews  should  be  kept  short  (less  than  one  hour)  as  constant 
interruption  and  repetitive  questioning  tends  to  disrupt  the  expert’s  train  of  thought 
and  can  inhibit  his  willingness  to  co-operate. 

Conversation  Theory  and  Focussed  Interviewing  are  similar  in  many  ways,  the  major 
features  of  the  latter  being  that  it  requires  the  interviewer  to  construct  a  list  of  topics 
for  discussion  so  that  they  can  be  sequentially  investigated.  Probing  enables  the 
interview  to  be  focussed  along  a  particular  thought  process  (similar  to  Conversation 
Theory's  two  level  analysis).  The  list  of  topics  can  be  either  investigated  in  a  breadth 
first  or  depth  first  approach.  A  depth  first  search  allows  the  knowledge  engineer  to 
gain  an  overall  high-level  understanding  of  the  domain.  This  is  contrary  to  the  natural 
thought  processes  applicable  to  natural  conversation  in  which  focussing  occurs  in  an 
associative  depth  first  manner.  Accordingly,  a  depth  first  focussed  interview  is  suited 
to  natural  conversation  and  generally  provides  better  results  than  the  breadth  first 
approach. 

3.4.  Problem  Solving 

Problem  Solving  requires  that  the  expert  is  given  a  problem  which  he  attempts  to  solve 
[Gammack84].  The  expert  describes  the  "what"  and  the  "why"  (for  each  goal/subgoal) 
until  the  problem  is  solved.  This  approach  requires  that  the  expert  fully  understands 
the  reasoning  at  each  step  as  well  as  being  able  to  relay  his  reasoning  to  the  knowledge 
engineer.  This  occasionally  leads  to  explanations  that  are  incongruous  with  general 
reasoning  given  by  the  same  expert  when  confronted  by  similar  problems.  As  a  result, 
numerous  problems  must  be  examined  in  order  to  reduce  the  probability  of  knowledge 
misrepresentation. 

3.5.  Repertory  Grids 

The  repertory  grid  approach  requires  the  expert  to  identify  a  set  of  domain  objects. 
Relationships  between  each  object  are  identified  by  exhaustively  selecting  the  objects 
in  sets  of  tlu^ee.  The  expert  then  explains  how  two  of  the  set  of  three  are  similar  and 
how  they  differ  from  the  third.  All  permutations  of  the  three  are  analysed  and  all 
permutations  of  objects  are  grouped  and  investigated. 
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This  approach  is  useful  for  gaining  an  understanding  of  the  relationships  between 
objects  in  a  specific  domain.  Unfortunately  the  repertory  grid's  Complexity  (C)  is 
greater  than  exponential  in  the  number  of  objects  (the  relationship  is  shown  in  equation 
1).  Consequently,  repertory  grids  become  unmanageable  for  domains  of  more  than 
approximately  10  objects  and  therefore  it  has  a  limited  use  and  is  an  impractical 
approach  for  the  majority  of  domains. 

C  =  n  ! 

2  X  (n  -  3)  - (1) 

where  n  is  the  number  of  objects 
in  the  set  and  n  >  3. 

4  Knowledge  Acquisition  by  Induction 

When  a  knowledge  engineer  participates  in  the  process  of  knowledge  acquisition  he  is 
basically  serving  an  apprenticeship.  In  practice,  apprenticeships  generally  require  a  human 
to  spend  a  number  of  years  learning  by  experience  and  structured  courses.  The  structured  courses 
give  the  apprentice  the  background  theory  of  a  domain,  whereas  the  experience  gained  by 
solving  numerous  problems  eventually  leads  to  the  apprentice  becoming  an  expert. 

Another  example  of  learning  by  problem  solving  is  that  of  undergraduates  studying  for  a 
degree.  Lectures  give  the  student  the  background  theory  and  any  necessary  formulas,  whereas 
case  studies,  laboratory  sessions  and  tutorials  teach  the  student  how  to  apply  his  knowledge. 
Only  when  the  student  has  solved  enough  problems  will  he /she  be  able  to  pass  the  relevant 
examinations.  It  follows  therefore  that  if  case  studies  and  tutorials  are  used  by  a  knowledge 
elicitation  methodology,  then  it  may  be  possible  to  alleviate  some  of  the  difficulties 
associated  with  knowledge  acquisition. 

4.1  The  IDS  Algorithm 

Quinlan's  IDS  algorithm  [Quinlan83]  has  stimulated  research  and  commercial 
development  of  inductive  knowledge  acquisition  methodologies.  This  has  resulted  in 
packages  such  as  Expert-Ease  [Slocombe86],  PRISM  [Cendrowska87]  and  NEXTRA 
[Rappaport88]. 

This  inductive  approach  requires  the  exp)ert  to  initially  identify  a  set  of  attributes 
A={al,a2,...,an}.  Each  attribute  can  have  many  associated  values  V={vl,v2,....,Vm] 
that  are  defined  by  the  expert  (such  as  good,  bad  and  reasonable).  An  instance  of  the 
attribute  set  A  consists  of  a  unique  permutation  U  of  the  attribute  values.  For  each  U,  a 
conclusion  or  hypothesis  d  is  selected  and  inserted  by  the  expert.  Once  the  expert  has 
finished  inserting  all  of  the  possible  instance/hypothesis  test  cases  the  rules  are 
induced  under  the  following  assumptions: 

*  all  hypotheses  are  mutually  exclusive 

*  all  attribute  values  are  discrete 

’*■  each  attribute  is  relevant  to  the  set  A  in  order  to  generate  a  hypothesis  d 

*  all  hypotheses  are  correct 

The  above  assumptions  and  the  employed  inductive  algorithmic  approach  requires 
that  the  training  set  (test  case  set)  is  complete.  That  is,  every  permutation  of  attribute 
value  pairs  have  been  analysed  by  the  induction  algorithm.  If  this  is  not  accomplished 
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then  a  degradation  in  rule  set  confidence  will  result.  This  requirement  restricts  the  ID3 
approach  to  that  of  small  sets  of  A  and  small  numbers  of  attribute  values  in  which  the 
Size  (S)  of  the  complete  training  set  is  defined  by  equation  2.  For  example,  if  A  consists 
of  n  attributes  (n=10),  5  of  which  have  3  values  and  5  of  which  have  4  values,  then 
5=248,832.  This  set  size  is  far  too  large  for  an  expert  to  analyse  due  to  the  complexity  of 
the  task  in  which  n  decisions  have  to  be  made  for  each  instance  of  S.  The  Complexity 
(C)  of  the  demands  imposed  upon  the  expert  are  shown  in  equation  3,  from  which  the 
complexity  relationship  graph  (shown  in  figure  1)  is  derived.  This  graph  highlights 
the  impractical  demands  imposed  upon  the  expert's  analytical  skills,  the  exception 
being  in  donaains  such  as  medical  diagnostics  in  which  large  numbers  of  case  studies 
have  been  archived. 


n 


s=n 

i=l 


—  (2) 


n 

C  =  KxnxJ^  Vj  —  (3) 

i=l 

where  K  is  a  constant. 


The  1D3  rule  induction  algorithm  uses  entropy  functions  [Shannon49]  to  develop  a  rule 
decision  tree.  An  entropy  function  is  a  measure  of  the  information  yielded  by  an  event  or 
as  a  measure  of  how  unexpected  the  event  was.  In  this  approach  entropies  depend  only 
upon  the  probabilities  of  the  events  considered,  therefore  each  attribute  value  in  the 
set  A  is  considered  as  information  with  some  bits  of  information  deemed  more 
important  than  others.  Intuitively,  it  would  not  be  unreasonable  to  assume  that  the 
entropy  of  an  event  (H)  has  the  following  properties: 

H  is  nonnegative  Vpe]0,l]  (H(p)>0)  — (4) 


consequently  information  about  an  event  does  not  reduce  the  information  content 
already  known. 

Furthermore,  H  is  additive: 

Vp,q€]0,l]  (H(pq)  =  H(p)  +  h(q))  --(5) 


and  therefore  the  information  obtained  from  two  independent  events  is  the  sum  of 
the  information  yielded  by  the  two  single  events. 

Another  feature  of  H  is  that  it  is  normalised  so  that: 

H(l/2)  =  1  -—(6) 

this  supposition  (the  simple  alternative)  implies  that  if  an  event  has 
a  probability  of  occurrence  equal  to  1/2,  then  its  opposite  is  equally  probable. 
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The  above  assumptions  lead  to  Shannon’s  entropy  function  for  complete  probability 
distributions,  in  which  the  entropy  of  a  set  of  classifications  is  defined  as: 

H  =  -5;pOi)log2pOi)  --(7) 


where  p(di)  is  the  probability  of  hypothesis  di  occurring. 


ID3  uses  entropy  to  generate  a  rule  tree  structure  by  recursively  partitioning  the 
training  set  of  examples  into  a  number  of  subsets  of  A.  This  is  done  in  such  a  way  as  to 
maximise  H,  whereby  H=0  in  the  final  recursive  step.  This  systematic  approach 
ensures  that  each  partition  selects  the  attribute  that  yields  the  most  information, 
therefore  the  tree  is  created  in  which  maximum  information  yield  is  achieved  by  a 
decision  (attribute  value  pair)  defined  in  the  node  nearest  to  the  root  of  the  decision 
tree.  Accordingly,  the  ordered  insertion  of  decisions  based  upon  entropy  ensures  that  a 
minimal  decision  tree  is  formed  and  a  compact  set  of  rules  induced. 

A  typical  example  of  a  generated  decision  tree  structure,  consisting  of  attribute  value 
pairs  and  conclusions  is  shown  in  figure  2.  From  this  structure  the  following  rules  can  be 
derived; 


1)  al,vl  31, 

2)  al,v2  A  a2,vl  ^  32, 

3)  al,v2  A  a2,v2  ^31, 

4)  al,v3  A  a2,vl  a  a3,vl  32, 

5)  al,v3  A  a2,vl  a  a3,v2  33, 

6)  al,v3  A  a2,v2  31. 
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Figure  1.  Complexity  Relationships  of  Rule  Induction  Training  Sets 
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Figure  2.  Example  of  a  Decision  Tree  Structure 

4.2.  The  Disadvantages  of  the  ID3  Algorithm 

The  ID3  decision  tree  rule  induction  approach  is  a  computationally  efficient  tool  for 
deriving  rule-based  systems.  Unfortunately,  it  has  a  number  of  undesirable  features. 
Firstly,  the  quality  of  the  derived  rules  depends  upon  the  precision  and  completeness  of 
the  training  set  of  test  cases.  Secondly,  decision  trees  are  inherently  difficult  to 
analyse,  therefore  it  may  be  necessary  for  an  inference  engine  to  search  the  complete 
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structure  before  a  hypothesis  can  be  identified.  Finally,  the  major  limitation  of 
decision  trees  is  that  they  rely  upon  a  knowledge-base  consisting  of  a  single  tree 
structure.  This  is  often  an  inadequate  model  for  numerous  domains  of  expertise  and  as  a 
result,  redundant  rules  have  to  be  incorporated  into  the  tree.  Unfortunately,  this  leads 
to  excessively  large  knowledge-bases  and  instigates  the  generation  of  redundant 
questions.  For  example,  consider  the  rule  structure  of  figure  2,  if  the  ES  knows  that 
a2,v2  is  true  then  it  is  not  necessary  to  analyse  any  other  attribute  value  pairs  because 
al  and  dl  can  be  concluded  regardless  of  any  further  information.  With  this  decision 
tree  example  the  value  of  al  is  required  before  a  conclusion  is  reached  which  in  certain 
domains  can  lead  to  unnecessary,  costly  and  time  consuming  investigations. 

To  overcome  the  limitations  of  decision  tree  structures,  modifications  to  the  ID3 
algorithm  have  been  investigated.  One  approach,  PRISM  [Chendrowska87]  uses  a 
measure  of  information  gain  instead  of  entropy.  The  major  difference  of  this  modified 
IDS  algorithm  is  that  PRISM  identifies  only  relevant  attribute-value  pairs,  whereas 
ID,3  will  identify  the  most  relevant  attribute  and  utilise  each  of  its  values  even  if  one 
or  more  of  these  values  are  redundant.  Although  PRISM  provides  a  flexible  rule 
structure,  it  still  relies  upon  the  completeness  and  accuracy  of  the  training  set,  therefore 
rule  induction  techniques  have  a  limited  scope  for  practical  applications. 

5  Structtrred  Automated  Knowledge  Acquisition 

Automated  knowledge  acquisition  approaches  attempt  to  eliminate  the  need  for  highly 
skilled  knowledge  engineers.  This  is  accomplished  by  constraining  and  guiding  the  expert’s 
reasorung  processes  by  interactively  processing  formatted  information  offered  by  the  user,  so 
that  the  automated  acquisition  tool  can  store  the  knowledge  in  a  highly  specific 
representation  scheme.  This  ensures  that  the  acquired  knowledge  is  captured  in  a  format 
suitable  for  an  expert  system  shell  problem  solving  mechanism. 

The  foundation  of  the  automated  knowledge  acquisition  approach  is  that  of  identifying  the 
functions  of  a  domain's  knowledge  and  partitioning  it  into  a  number  of  functional  modules. 
This  enables  the  knowledge  engineer  to  guide  the  articulation  process,  as  opposed  to  asking 
unstructured  questions  which  rely  upon  the  co-operation  and  communication  skills  of  the 
expert.  Another  role  of  knowledge  partitioning  and  knowledge  function  identification  is  that 
of  explaining  to  the  user  the  knowledge  that  was  used  in  the  route  to  a  conclusion.  This  can  be 
considered  as  the  inverse  of  knowledge  acquisition  in  which  a  human  imparts  expertise  to  the 
system,  whereas  when  the  system  is  explaining  its  reasoning  it  is  imparting  knowledge  to  a 
human. 

After  the  function  of  a  knowledge  module  is  defined  and  understood,  it  is  possible  to  identify 
any  incomplete  or  missing  associated  areas.  Furthermore,  the  functionally  partitioned 
modular  knowledge-base  structure,  in  which  each  partitioned  block  is  allocated  a  task  or  set 
of  tasks,  simplifies  the  design,  debugging  and  evaluation  of  the  evolving  ES. 

5.1  MORE 

Over  the  past  decade  there  have  been  many  reported  automatic  knowledge  acquisition 
tools  such  as  TIERESIAS  [Davis82]  ,  ETS  [Boose84],  MORE  [Kahn85]  and  MOLE 
[Eshelman87].  During  an  interactive  knowledge  acquisition  session  these  tools 
generally  construct  the  knowledge-base  in  a  step  by  step  process.  At  each  stage  the 
acquired  knowledge  becomes  more  complete  therefore  allowing  the  tool  to  focus  upon 
relevant  questions.  Inconsistencies  are  also  highlighted  and  modified  as  required,  for 
example,  MORE  is  founded  upon  a  three  stage  process  in  which  stage  one  is  a  simple 
acquisition  of  observations  and  their  corresponding  hypotheses.  Whilst  the  expert  is 
inserting  these  relationships  MORE  will  request  further  information  so  that  the 
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acquired  knowledge  will  be  of  a  specific  nature.  At  this  stage  a  certainty  factor 
associated  with  the  individual  rules  may  be  included. 

The  second  phase  of  MORE's  knowledge  acquisition  process  is  the  analysis  of  the 
knowledge-base  that  was  constructed  in  phase  one.  This  involves  the  evaluation  of 
each  hypotheses'  certainty  factor.  If  a  hypothesis  occurs  only  once  in  the  knowledge¬ 
base  and  its  certainty  factor  is  neither  for  or  against  the  hypothesis,  then  this 
hypothesis  is  investigated  further  by  MORE.  Hence,  meaningless  or  vague  hypotheses 
are  elinunated  and  redundant  rules  removed  from  the  knowledge-base. 

In  the  third  automated  acquisition  step,  inconsistencies  are  investigated  so  that  the 
certainty  factors  associated  with  highly  specific  rules  have  stronger  weights  than  less 
specific  ones.  For  example,  consider  the  following  two  rules:- 

If  A  and  B  and  C  then  D. 

If  C  then  E. 

The  certainty  factor  associated  with  the  first  rule  should  be  higher  than  that  of  the 
second  rule,  if  not,  there  has  probably  been  an  error  in  the  expert's  reasoning  during 
phase  one  or  two.  MORE  checks  for  these  inconsistencies  and  reports  them  to  the  expert 
so  that  modifications  can  take  place. 

The  designers  of  these  automated  acquisition  approaches  have  claimed  that  they  have 
practically  eliminated  the  "Feigenbaum  Bottleneck"  [Feigenbaum79]  and  have 
dramatically  reduced  the  need  for  highly  experienced  knowledge  engineers.  In  practice 
this  is  not  the  case  and  although  these  tools  are  a  step  in  the  right  direction,  further 
research  is  required  to  improve  their  performance;  once  this  is  achieved,  it  may  be 
possible  to  use  these  tools  to  rapidly  acquire  highly  specific  domain  dependant 
knowledge.  This  will  only  be  possible  if  a  knowledge  engineer  is  involved  in  the 
preliminary  and  final  stages  of  the  knowledge  acquisition  process. 

6  Conclusion  and  future  directions 

There  are  many  problems  associated  with  knowledge  acquisition.  For  example,  articulation 
approaches  are  iterative  and  require  highly  skilled  interviewing  techniques  so  as  to 
maximise  the  interactions  between  the  knowledge  engineer  and  domain  expert. 

In  contrast  to  articulation  approaches,  rule  induction  techniques  are  non-iterative  but  are 
generally  only  applicable  to  small  sets  of  relationships,  the  exception  being  when  large 
amounts  of  case  studies  are  accessible.  Furthermore,  unless  an  exhaustive  training  set  is  used 
other  acquisition  techniques  may  be  required  to  refine  the  set  of  induced  rules. 

Automatic  knowledge  acquisition  methodologies  are  becoming  more  evident  in  ES  construction. 
Unfortunately,  they  require  the  domain  of  expertise  to  be  modelled  by  highly  specific 
formations  (rule-based  structures  with  a  probabilistic  measure  of  belief).  Currently 
automated  approaches  have  limited  capabilities  and  they  do  not  explicitly  utilise 
hierarchical  relationships  or  object  oriented  structures.  Thus,  if  these  features  are 
incorporated  into  an  automatic  acquisition  tool,  driven  by  "question  and  answer"  interactions, 
then  highly  structured  knowledge  schemas  could  be  created.  Such  an  approach  could  be 
greatly  facilitated  by  an  integrated  software  architecture  consisting  of  a  new  hierarchical 
knowledge  representation  schema.  This  would  be  implemented  by  the  use  of  a  graphical 
interface  and  a  constrained  natural  language  processor.  As  a  result,  techniques  applicable  to 
graphical  modelling  and  application  specific  natural  language  processing  must  investigated. 
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